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Background 


• Essays written by individuals from different 
language backgrounds often differ in their 
linguistic, stylistic, and rhetorical characteristics 
and these features may affect scores. 

• Many studies have documented differences in the 
composing processes, features, and structure of 
compositions by LI and L2 writers. 


0 


CollegeBoard 

inspiring minds' 



Background, continued 


• Research has shown that in general L2 students: 

• Write at a slower rate, and have shorter texts with more 
errors, compared to LI texts. 

• Use a different argument structure than LI writers, and 
they less often state and support their position fully. 

• Employ simple sentences with more personal references 
and repetitions of ideas and vocabulary. 

• Exhibit more use of the first person singular, or the 
pronoun “I”. 
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Background, continued 


Variation in task requirements and instructions significantly 
influences the characteristics of L2 essays. For example: 

• Hinkel (2009): broad topics require L2 writers to rely on 
their own experience in lieu of factual evidence, leading 
to overuse of certain language features (obligation and 
necessity modals). 

• Hirokawa & Swales (1986): L2 essays written to a simple 
topic were longer, contained more subordination, and 
exhibited greater use of the first-person than those 
written to an academic topic. 
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Research Questions 


• Do LI , L2, and bilingual SAT-takers differ in the 
frequency of use of particular essay features? 

• Are there differences in the frequency of use of 
essay features for language groups within ethnic 
groups (e.g., Asian, Hispanic)? 

• Is there variability in the use of essay features by 
language group across different essay prompts? 
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Method 


• Data source - The SAT essay, a 20-minute holistically- 
scored essay. Essays were written to 14 prompts 
administered between March 2005 and January 2006 
(n=6,570). Students’ best language and ethnicity were 
taken from the SAT Questionnaire. 

• Essays were coded for a variety of features, with reliability 
(percent agreement) ranging from .69 to .98. 

• Analyses included descriptive statistics (frequencies of 
essay features by language within ethnic group), 
standardized differences (effect sizes), and logistic 
multilevel regression (HGLM). 
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Coded Essay Features and Reliability 


Essay 

Feature 

% 

agreement 

Type of 
Examples 

% 

agreement 

Number of words 

.90* 

Academic 

88 

Use of first-person 

96 

Politics/Professions 

84 

Concludes mid-sentence 

95 

Popular Culture/Sports 

89 

Five-paragraph theme 

92 

Personal experience 

77 

Quoted stimulus 

69 

Religious 

98 

Rhetorical approach 

83 

No example 

98 


* Pearson correlation 


CdlegeBoard 

inspiring minds' 









The Five-Paragraph Theme 



Borrowed from www.sparknotes.com 
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Results: Standardized Differences 


Std. Diff. 

Asian 

Af. American 

Hispanic 

White 

LI vs. L2 





SAT Essay 

0.78 

0.31 

0.71 

0.38 

SAT-W 

0.83 

0.36 

0.85 

0.47 

N words 

0.36 

0.18 

0.29 

0.11 

LI vs. BIL 





SAT Essay 

0.17 

0.00 

0.06 

0.00 

SAT-W 

0.23 

0.06 

0.15 

0.07 

N words 

-0.12 

-0.08 

0.02 

-0.16 


Standardized differences were computed by subtracting the means for L2 and BIL 
from the means for LI and dividing by the total standard deviation for each ethnic 
group. 
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Results: Variability in Frequency of 
Selected Essay Features Across Prompts 



Essay Feature 

LI 

BIL 

L2 

First-person 

.35 - .62 

.33 - .78 

.43 - .74 

Concluded mid-sentence 

0 

CO 

1 

00 

. 10 - .24 

.09 - .24 

Five-paragraph theme 

. 04 - .14 

. 04 - .15 

.01 - .14 

Quoted stimulus 

. 10 - .45 

.07 - .57 

.05 - .45 

Personal experience 

.23 - .78 

. 18 - .76 

.34 - .87 

Academic example 

. 16 - .88 

. 18 - .94 

.09 - .81 

Rhetorical approach 3 
Agreed 

.05 - .92 

. 16 - .93 

.03 - .90 

Disagreed 

.01 - .24 

.01 - .22 

.01 - .28 

Agreed & disagreed 

.05 - .94 

.04 - .97 

.04 - .90 

No position 

.00 - .08 

.00 - . 04 b 

. 00 - .13 


a Rhetorical approach was coded for 10 of the 14 prompts. b There were only 12 BIL 
students taking no position across all prompts. 
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Results: Variability in Frequency of 
Selected Essay Features Across Prompts 


Essay Feature 

LI 

BIL 

L2 

First-person 

.35 - .62 

.33 - .78 

.43 - .74 

Concluded mid-sentence 

0 

CO 

1 

00 

. 10 - .24 

.09 - .24 

Five-paragraph theme 

. 04 - .14 

. 04 - .15 

.01 - .14 

Quoted stimulus 

. 10 - .45 

.07 - .57 

.05 - .45 

Personal experience 

.23 - .78 

. 18 - .76 

.34 - .87 
f .09 - . 8 l "| 

Academic example 

. 16 - .88 

. 18 - .94 

Rhetorical approach 3 
Agreed 

.05 - .92 

. 16 - .93 

[.03 - .90 

Disagreed 

.01 - .24 

.01 - .22 

.01 - .28 

Agreed & disagreed 

.05 - .94 

r 04-.97 

.04 - .90 

No position 

.00 - .08 

.00 - . 04 b 

.00 - .13 


a Rhetorical approach was coded for 10 of the 14 prompts. b There were only 12 BIL 
students taking no position across all prompts. 
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“The Working Life” Prompt 


It’s easy to see why - aside from the income it provides - having a job is 
so desirable in our culture. Work works for us. It structures our time 
and imposes a rhythm on our lives. It gets us organized into various 
kinds of communities and social groups. And perhaps most important, 
work tells us what to do every day. 

Adapted from Joanne B. Ciulla, “The Working Life” 

Assignment: Do people depend on work - whether it is a job, 
schoolwork, or volunteer work - to determine what their daily activities 
and interactions with others should be? Plan and write an essay in 
which you develop your point of view on this issue. Support your 
position with reasoning and examples taken from your reading, studies, 
experience, or observations. 
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Results: Multilevel Logistic Regression 
Odds Ratios 


Essay Feature 

Intercept 

Writing 

MC 

BIL 

L2 

Asian 

Black 

Concluded mid-sent. 

0.13 

0.99 

0.95 

1.06 

1.60 

1.37 

First-person 

1.04 

0.97 

1.06 

1.27 


1.01 

1.16 

Five-paragraph theme 

0.06 

1.05 

1.62 

0.59 

1.92 

1.04 

Quoted stimulus 

0.39 

0.99 

1.07 

1.35 


0.91 

0.86 

Personal experience 

1.08 

0.99 

0.96 

1.23 

0.92 

1.03 

Academic example 

0.87 

1.06 

1.20 

1.12 


1.40 

0.95 

Rhetorical approach 








Agreed 

1.11 

1.01 

1.20 

1.19 


1.08 

1.19 

Disagreed 

0.12 

0.99 

0.79 1.02 


0.87 

0.81 

Agreed & Disagreed 

0.35 

1.01 

1.01 

0.74 

1 

0.98 

0.90 

No position 

0.01 

0.94 

0.68 

1.08 


1.19 

1.35 

Interaction Effects 

BIL x Asian 

BIL x 

BIL x 

L2x 


L2 x 

L2 x 



Black 

Hisp. i 

Asian 


Black 

Hisp. 

Five-paragraph theme 

0.51 

1.25 

0.90 

1.97 

2.73 

3.46 

Quoted stimulus 

1.04 

1.43 

0.87 

0.52 

1 1 

0.50 

0.50 


Hispanic 


1.45 

0.85 

0.72 

1.20 

1.00 

0.90 

1.14 

0.88 

0.89 

1.35 

























Concluded mid-sentence 


0.01 


( 0 . 10 ) 







First-person 


0.12(0.34) 


Five-paragraph theme 


0.08 (0.28) 


Quoted stimulus 


0.34 (0.58) 


Personal experience 


0.66 (0.82) 


Academic example 
Rhetorical approach 
Agreed 
Disagreed 


1.73(1.32) 



0.15(0.39) 



2.52(1.59) 


< 0.01 (0.02) 




0.89 (0.94) 



Agreed & disagreed 


2.66(1.63) 


No position 


0.80 (0.90) 





**p< .01 *p<.05. Dashes indicate that final model fixed slopes for these variables. 
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Conclusions and Discussion 


• This study adds to the literature finding that L2 students 
differ in the nature of their writing in response to essay 
tests. 

• L2 students were more likely to use first-person and 
personal experience, and were less likely to take a 
mixed argument approach when responding to the SAT 
essay prompts. 

• This study examined the use of each essay feature 
independently although we know that test-takers use a 
combination of many of these features. 
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Conclusions and Discussion, continued 


• There was substantial variability in the frequency of essay 
features by language group across the 1 4 prompts. 

• Characteristics of the prompts to help explain this 
variability were not examined in this study, but will be 
important to explore in future research. 

• Silva (1993) noted that the difference between LI and L2 
writers “needs to be acknowledged and addressed by 
those who deal with L2 writers if these writers are to be 
treated fairly, taught effectively, and thus, given an equal 
chance to succeed in their writing-related personal and 
academic endeavors.” 
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Thank You! 


College Board researchers are encouraged 
to freely express their professional judgment. 
Therefore, the points of view or opinions 
stated in this presentation do not necessarily 
represent official College Board position or 
policy. 

Please forward any questions, comments, 
and suggestions to Jennifer Kobrin at 
jkobrin@collegeboard.org. 
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